Effects of variability in models: a family of experiments

The ever-growing need for customization creates a need to maintain software systems in many different variants. To avoid having to maintain different copies of the same model, developers of modeling languages and tools have recently started to provide implementation techniques for such variant-rich systems, notably variability mechanisms, which support implementing the differences between model variants. Available mechanisms either follow the annotative or the compositional paradigm, each of which have dedicated benefits and drawbacks. Currently, language and tool designers select the used variability mechanism often solely based on intuition. A better empirical understanding of the comprehension of variability mechanisms would help them in improving support for effective modeling. In this article, we present an empirical assessment of annotative and compositional variability mechanisms for three popular types of models. We report and discuss findings from a family of three experiments with 164 participants in total, in which we studied the impact of different variability mechanisms during model comprehension tasks. We experimented with three model types commonly found in modeling languages: class diagrams, state machine diagrams, and activity diagrams. We find that, in two out of three experiments, annotative technique lead to better developer performance. Use of the compositional mechanism correlated with impaired performance. For all three considered tasks, the annotative mechanism was preferred over the compositional one in all experiments. We present actionable recommendations concerning support of flexible, tasks-specific solutions, and the transfer of established best practices from the code domain to models

Software variability in service robotics

Robots artificially replicate human capabilities thanks to their software, the main embodiment of intelligence. However, engineering robotics software has become increasingly challenging. Developers need expertise from different disciplines as well as they are faced with heterogeneous hardware and uncertain operating environments. To this end, the software needs to be variable—to customize robots for different customers, hardware, and operating environments. However, variability adds substantial complexity and needs to be managed—yet, ad hoc practices prevail in the robotics domain, challenging effective software reuse, maintenance, and evolution. To improve the situation, we need to enhance our empirical understanding of variability in robotics. We present a multiple-case study on software variability in the vibrant and challenging domain of service robotics. We investigated drivers, practices, methods, and challenges of variability from industrial companies building service robots. We analyzed the state-of-the-practice and the state-of-the-art—the former via an experience report and eleven interviews with two service robotics companies; the latter via a systematic literature review. We triangulated from these sources, reporting observations with actionable recommendations for researchers, tool providers, and practitioners. We formulated hypotheses trying to explain our observations, and also compared the state-of-the-art from the literature with the-state-of-the-practice we observed in our cases. We learned that the level of abstraction in robotics software needs to be raised for simplifying variability management and software integration, while keeping a sufficient level of customization to boost efficiency and effectiveness in their robots’ operation. Planning and realizing variability for specific requirements and implementing robust abstractions permit robotic applications to operate robustly in dynamic environments, which are often only partially known and controllable. With this aim, our companies use a number of mechanisms, some of them based on formalisms used to specify robotic behavior, such as finite-state machines and behavior trees. To foster software reuse, the service robotics domain will greatly benefit from having software components—completely decoupled from hardware—with harmonized and standardized interfaces, and organized in an ecosystem shared among various companies

EMMM: A Unified Meta-Model for Tracking Machine Learning Experiments

Traditional software engineering tools for managing assets—specifically, version control systems—are inadequate to manage the variety of asset types used in machine-learning model development experiments. Two possible paths to improve the management of machine learning assets include 1) Adopting dedicated machine-learning experiment management tools, which are gaining popularity for supporting concerns such as versioning, traceability, auditability, collaboration, and reproducibility; 2) Developing new and improved version control tools with support for domain-specific operations tailored to machine learning assets. As a contribution to improving asset management on both paths, this work presents Experiment Management Meta-Model (EMMM), a meta-model that unifies the conceptual structures and relationships extracted from systematically selected machine-learning experiment management tools. We explain the metamodel’s concepts and relationships and evaluate it using real experiment data. The proposed meta-model is based on the Eclipse Modeling Framework (EMF) with its meta-modeling language, Ecore, to encode model structures. Our meta-model can be used as a concrete blueprint for practitioners and researchers to improve existing tools and develop new tools with native support for machine-learning-specific assets and operations

A benchmark generator framework for evolving variant-rich software

Software often needs to exist in different variants, which account for varying customer requirements, environments, or non-functional aspects, such as energy consumption. Unfortunately, the number of variants can grow exponentially with the number of features. As such, developing and evolving variant-rich systems is challenging, since they do not only evolve “in time” as single systems, but also “in space” with new variants. Fortunately, many different methods and tools for variant-rich systems have been proposed over the last decades, especially in the field of software product line engineering. However, their level of evaluation varies significantly, threatening their relevance for practitioners and that of future research. Many tools have only been evaluated on ad hoc datasets, minimal examples, or unrealistic and limited evolution scenarios, missing large parts of the actual evolution lifecycle of variant-rich systems. Our long-term goal is to provide benchmarks to increase the maturity of evaluation of methods and tools for evolving variant-rich systems. However, providing manually curated and sufficiently detailed benchmarks that cover the whole evolution lifecycle of variant-rich systems is challenging. We present the framework vpbench, which simulates the evolution of a variant-rich system and thereby generates an evolution enriched with metadata explaining the evolution. The generated benchmarks, i.e., the evolution histories and metadata, can serve as ground truth to check the results of tools applied on it. We formalize the claims we make about the generator and the generated benchmarks as requirements. The design of vpbench comprises modular generators and evolution operators that automatically evolve real codebases. We implement simple and advanced evolution operators—e.g., relying on code transplantation to incorporate features from real projects. We demonstrate how vpbench addresses its claimed requirements, also considering multiple degrees of realism, extensibility and language-independence of the generated benchmarks

Exploring conflict reasons for graph transformation systems

Conflict and dependency analysis (CDA) is a static analysis for the detection of conflicting and dependent rule applications in a graph transformation system. Recently, granularity levels for conflicts and dependencies have been investigated focussing on delete-use conflicts and produce-use dependencies. A central notion for granularity considerations are (minimal) conflict and dependency reasons. For a rule pair, where the second rule is non-deleting, it is well-understood based on corresponding constructive characterizations how to efficiently compute (minimal) conflict and dependency reasons. We further explore the notion of (minimal) conflict reason for the general case where the second rule of a rule pair may be deleting as well. We present new constructive characterizations of (minimal) conflict reasons distinguishing delete-read from delete-delete reasons. Based on these constructive characterizations we propose a procedure for computing (minimal) conflict reasons and we show that it is sound and complete

Detecting security vulnerabilities using clone detection and community knowledge

Faced with the severe financial and reputation implications associated with data breaches, enterprises now recognize security as a top concern for software analysis tools. While software engineers are typically not equipped with the required expertise to identify vulnerabilities in code, community knowledge in the form of publicly available vulnerability databases could come to their rescue. For example, the Common Vulnerabilities and Exposures Database (CVE) contains data about already reported weaknesses. However, the support with available examples in these databases is scarce. CVE entries usually do not contain example code for a vulnerability, its exploit or patch. They just link to reports or repositories that provide this information. Manually searching these sources for relevant information is time-consuming and error-prone. In this paper, we propose a vulnerability detection approach based on community knowledge and clone detection. The key idea is to harness available example source code of software weaknesses, from a large-scale vulnerability database, which are matched to code fragments using clone detection. We leverage a clone detection technique from the literature, which we adapted to make it applicable to vulnerability databases. In an evaluation based on 20 reports and affected projects, our approach showed good precision and recall

Message from the MiSE 2019 Workshop Organizers

No abstract availabl

Finding the\ua0Right Way to Rome: Effect-Oriented Graph Transformation

Many applications of graph transformation require rules that change a graph without introducing new consistency violations. When designing such rules, it is natural to think about the desired outcome state, i.e., the desired effect, rather than the specific steps required to achieve it; these steps may vary depending on the specific rule-application context. Existing graph-transformation approaches either require a separate rule to be written for every possible application context or lack the ability to constrain the maximal change that a rule will create. We introduce effect-oriented graph transformation, shifting the semantics of a rule from specifying actions to representing the desired effect. A single effect-oriented rule can encode a large number of induced classic rules. Which of the potential actions is executed depends on the application context; ultimately, all ways lead to Rome. If a graph element to be deleted (created) by a potential action is already absent (present), this action need not be performed because the desired outcome is already present. We formally define effect-oriented graph transformation, show how matches can be computed without explicitly enumerating all induced classic rules, and report on a prototypical implementation of effect-oriented graph transformation in Henshin

Model-Driven Optimization: Generating Smart Mutation Operators for Multi-Objective Problems

In search-based software engineering (SBSE), the choice of search operators can significantly impact the quality of the obtained solutions and the efficiency of the search. Recent work in the context of combining SBSE with model-driven engineering has investigated the idea of automatically generating smart search operators for the case at hand. While showing improvements, this previous work focused on single-objective optimization, a restriction that prohibits a broader use for many SBSE scenarios. Furthermore, since it did not allow users to customize the generation, it could miss out on useful domain knowledge that may further improve the quality of the generated operators. To address these issues, we propose a customizable framework for generating mutation operators for multi-objective problems. It generates mutation operators in the form of model transformations that can modify solutions represented as instances of the given problem meta-model. To this end, we extend an existing framework to support multiobjective problems as well as customization based on domain knowledge, including the capability to specify manual "baseline" operators that are refined during the operator generation. Our evaluation based on the Next Release Problem shows that the automated generation of mutation operators and user-provided domain knowledge can improve the performance of the search without sacrificing the overall result quality

Multi-granular conflict and dependency analysis in software engineering based on graph transformation (Summary)

No abstract availabl